Virtual optical touch screen detecting touch distance

ABSTRACT

Apparatus and methods are provided for converting a display into a touch display. One or more optical sensors are arranged proximate the display. A processor receives a signal from the optical sensor(s) and, based on the signal, executes instructions for: (i) determining a distance between a pointer device and a surface of the display signal; (ii) determining a position of the pointer device on the surface of the display; (iii) when the distance is less than a threshold detection distance and greater than a threshold touch distance, providing an indication on the display that the position of the pointer device has been determined; and (iv) when the distance is less than the threshold touch distance, taking an action indicative of contact between the pointer device and the surface of the display.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of, and incorporatesherein by reference in its entirety, U.S. Provisional Patent ApplicationNo. 61/811,368, entitled “Virtual Touch Screen,” which was filed on Apr.12, 2013.

FIELD OF THE INVENTION

Embodiments of the invention relate generally to displays and, moreparticularly, to apparatus and methods for converting a display into atouch display.

BACKGROUND

Graphical displays may be categorized as being either touch displays ornon-touch displays. With touch displays, a user may provide input to anassociated processor or computing device by contacting the display(e.g., with a finger) to select or manipulate an item on the display.With non-touch displays, a user is unable to interact directly with thedisplay and must provide input to the associated processor using aseparate input device, such as a mouse or keyboard. Wireless motionsensors allow users to interact with displays via skeletal sensing andother means, but require the user to be a certain distance (e.g., morethan 3 meters) away from the display.

There is a need for apparatus and methods that allow a non-touch displayto be converted into a touch display and that allow the user to operatein close proximity to the display.

SUMMARY OF THE INVENTION

Embodiments of the apparatus and method described herein utilize one ormore optical sensors to allow a non-touch display to behave like a touchdisplay. In particular, an optical sensor may be used to detect theposition of a user's finger or other pointer device with respect to thedisplay. When the user moves the finger to within a threshold detectiondistance (e.g., 1 to 3 inches, or less than about 6 inches) of thedisplay, the display may provide an indication that the position of thefinger has been detected. For example, an item (e.g., a button) that theuser could select may become brighter to let the user know that the itemis available for selection. When the user moves the finger closer to thedisplay, such that the finger is within a threshold touch distance(e.g., less than about 1 inch) of the display, the apparatus may take anaction that is consistent with direct contact between the finger and thesurface of the display. For example, the action may indicate that anitem has been selected by the user. Alternatively or additionally, theaction may indicate that the user is manipulating or modifying an imageon the display, such as translating or expanding the image, or drawinglines or other features in the image.

Advantageously, the apparatus and methods described herein allow anynon-touch display to be converted into or used as a touch display.Suitable non-touch displays that may be used with the apparatus andmethods include, for example, flat screen televisions, projections on ascreen or wall from a projector, and computer monitors.

In one aspect, the invention relates to an apparatus for converting adisplay into a touch display. The apparatus includes at least oneoptical sensor arranged proximate the display and having at least onedepth sensor. The apparatus also includes a processor configured toreceive a signal from the at least one optical sensor. The processor isconfigured to execute instructions for: (i) determining a distancebetween at least one pointer device (e.g., one or more fingers of auser) and a surface of the display, based on the signal; (ii)determining a position of the at least one pointer device on the surfaceof the display, based on the signal; (iii) when the distance is lessthan a threshold detection distance (e.g., less than about 6 inches) andgreater than a threshold touch distance (e.g., less than about 1 inch),providing an indication on the display that the position of the at leastone pointer device has been determined; and (iv) when the distance isless than the threshold touch distance, taking an action indicative ofcontact between the at least one pointer device and the surface of thedisplay.

In certain embodiments, the at least one optical sensor is oriented in adirection that is substantially parallel to the surface of the display.The at least one optical sensor may also include at least one camera,and/or the at least one depth sensor may include a laser projector. Insome embodiments, the at least one optical sensor includes a firstoptical sensor oriented in a first direction, and a second opticalsensor oriented in a second direction, such that the first direction issubstantially orthogonal to the second direction, and the first andsecond directions are substantially parallel to the surface of thedisplay. The at least one optical sensor may also include a thirdoptical sensor oriented in a third direction, and a fourth opticalsensor oriented in a fourth direction, such that the third direction issubstantially orthogonal to the fourth direction, and the third andfourth directions are substantially parallel to the surface of thedisplay.

In certain embodiments, the at least one pointer device includes two ormore pointer devices, and the processor is configured to determine aposition of each pointer device on the surface of the display. In someembodiments, determining a position includes determining a closest pointto the at least one pointer device on the surface of the display.Providing an indication on the display may include adjusting an image onthe display in the vicinity of the position of the at least one pointerdevice on the surface of the display. Taking an action indicative ofcontact between the at least one pointer device and the surface of thedisplay may include, for example, translating an image, expanding animage, shrinking an image, selecting an item or object, and/or rotatinga virtual object.

In another aspect, the invention relates to a method of converting adisplay into a touch display. The method includes: receiving a signalfrom at least one optical sensor arranged proximate the display andhaving at least one depth sensor; determining a distance between atleast one pointer device (e.g., one or more fingers of a user) and asurface of the display, based on the signal; determining a position ofthe at least one pointer device on the surface of the display, based onthe signal; when the distance is less than a threshold detectiondistance (e.g., less than about 6 inches) and greater than a thresholdtouch distance (e.g., less than about 1 inch), providing an indicationon the display that the position of the at least one pointer device hasbeen determined; and when the distance is less than the threshold touchdistance, taking an action indicative of contact between the at leastone pointer device and the surface of the display.

In certain embodiments, the at least one optical sensor is oriented in adirection that is parallel or substantially parallel to the surface ofthe display. The at least one optical sensor may include at least onecamera, and/or the at least one depth sensor may include a laserprojector. In some embodiments, the at least one optical sensor includesa first optical sensor oriented in a first direction, and a secondoptical sensor oriented in a second direction, such that the firstdirection is substantially orthogonal to the second direction, and thefirst and second directions are substantially parallel to the surface ofthe display. The at least one optical sensor may also include a thirdoptical sensor oriented in a third direction, and a fourth opticalsensor oriented in a fourth direction, such that the third direction issubstantially orthogonal to the fourth direction, and the third andfourth directions are substantially parallel to the surface of thedisplay.

In certain embodiments, the at least one pointer device includes two ormore pointer devices, and determining a position of the least onepointer device includes determining a position of each pointer device onthe surface of the display. Determining a position may includedetermining a closest point to the at least one pointer device on thesurface of the display. Providing an indication on the display mayinclude adjusting an image on the display in the vicinity of theposition of the at least one pointer device on the surface of thedisplay. In some embodiments, taking an action indicative of contactbetween the at least one pointer device and the surface of the displayincludes selecting an item, translating an image, expanding an image,shrinking an image, selecting an item or object, and/or rotating avirtual object.

Elements of embodiments described with respect to a given aspect of theinvention may be used in various embodiments of another aspect of theinvention. For example, it is contemplated that features of dependentclaims depending from one independent claim can be used in apparatusand/or methods of any of the other independent claims.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects and features of the invention can be better understood withreference to the drawings described below, and the claims. The drawingsare not necessarily to scale, emphasis instead generally being placedupon illustrating the principles of the invention. In the drawings, likenumerals are used to indicate like parts throughout the various views.

While the invention is particularly shown and described herein withreference to specific examples and specific embodiments, it should beunderstood by those skilled in the art that various changes in form anddetail may be made therein without departing from the spirit and scopeof the invention.

FIG. 1 is a schematic front view of an optical sensor positioned near adisplay, according to an illustrative embodiment of the invention.

FIG. 2 is a schematic front view of two optical sensors positioned nearadjacent corners of a display, according to an illustrative embodimentof the invention.

FIG. 3 is a schematic front view of four optical sensors, eachpositioned near a corner of a display, according to an illustrativeembodiment of the invention.

FIG. 4 is an image obtained from an optical sensor positioned beneath adisplay, showing a user's finger in close proximity to the display,according to an illustrative embodiment of the invention.

FIG. 5 includes two stacked images obtained from an optical sensorpositioned beneath a display, showing a user's finger in close proximitywith corners of a display, according to an illustrative embodiment ofthe invention.

FIGS. 6 and 7 are screenshots of a calibration utility used to configurean optical sensor with respect to a display, according to anillustrative embodiment of the invention.

FIG. 8 is a schematic diagram of an optical sensor used to determine aposition of a user's finger relative to a display, according to anillustrative embodiment of the invention.

FIG. 9 is a flowchart of a method for converting a display into a touchdisplay, according to an illustrative embodiment of the invention

DETAILED DESCRIPTION

It is contemplated that apparatus, systems, methods, and processes ofthe claimed invention encompass variations and adaptations developedusing information from the embodiments described herein. Adaptationand/or modification of the apparatus, systems, methods, and processesdescribed herein may be performed by those of ordinary skill in therelevant art.

Throughout the description, where devices and systems are described ashaving, including, or comprising specific components, or where processesand methods are described as having, including, or comprising specificsteps, it is contemplated that, additionally, there are devices andsystems of the present invention that consist essentially of, or consistof, the recited components, and that there are processes and methodsaccording to the present invention that consist essentially of, orconsist of, the recited processing steps.

It should be understood that the order of steps or order for performingcertain actions is immaterial so long as the invention remains operable.Moreover, two or more steps or actions may be conducted simultaneously.

Referring to FIG. 1, in various embodiments, apparatus and methods areprovided that use an optical sensor 10 to convert a non-touch displayinto a touch display. The optical sensor 10 is positioned proximate adisplay 12 and oriented in a direction that is substantially parallel toa surface of the display. For example, the optical sensor 10 may bepositioned a distance D of about 1 meter below a bottom edge of thedisplay 12. A field of view 14 of the optical sensor is preferably wideenough to encompass the entire display 12. The optical sensor 10 mayinclude one or more depth sensors (e.g., laser projectors) that allow aposition of a user's finger 16 or other pointing device to be determinedrelative to a surface of the display 12. For example, the depthsensor(s) may be used to determine a distance between the optical sensor10 and the user's finger (e.g., along the surface of the display). Theoptical sensor 10 may also include one or more cameras for obtaining animage of the user's finger 16 near the display. In some embodiments, theoptical sensor 10 is a KINECT™ device, available from MICROSOFT® ofRedmond, Wash. Alternatively, the optical sensor 10 may be a PRIMESENSEOPENNI 3D sensor, available from PrimeSense, LTD.

The display may be any type of display capable of presenting an image.Suitable displays include, for example, computer monitors andtelevisions (e.g., flat screen HD televisions). In some embodiments, thedisplay is a wall, a floor, a screen, a table, a desk, or other surfacedisplaying an image projected from behind or in front of the surfaceusing, for example, a video projector. The display surface is preferablysmooth and flat, although rough, curved, and/or textured surfaces mayalso be used.

In various embodiments, the apparatus and methods described herein usedata from the optical sensor 10 to determine a position of the user'sfinger 16 with respect to the display 12 while allowing the user to bepositioned in close proximity to the display 12 and interact directlywith the display 12. For example, the apparatus and methods maydetermine when the user's finger 16 is within a threshold detectiondistance from a surface of the display 12. The threshold detectiondistance may be, for example, less than about 12 inches, less than about6 inches, less than about 3 inches, or less than about 1.5 inches. Whenthe user's finger 16 is within the threshold detection distance from thedisplay, the apparatus and methods may use the optical sensor 10 todetermine a position on the display 12 that corresponds to the user'sfinger 16. The position may be a point on the surface of the display 12that is closest to the user's finger 16 and/or corresponds to aprojection of the user's finger 16 onto the surface of the display 12.Once the position of the finger 16 has been identified, the apparatusand methods may provide an indication on the display 12 that theposition of the finger 16 has been determined. For example, a cursor mayappear at the position of the finger 16. Alternatively, a button in thevicinity of the user's finger 16 may become highlighted to inform theuser that the button is available for selection. In one embodiment, theapparatus and methods provide a circle 18 or other icon (e.g., a cursoravatar) on the display 12 around the position of the user's finger 16,when the finger 16 is hovering in front of the display 12. A diameter orarea of the circle 18 (or size of the cursor avatar) may be proportionalor otherwise related to the distance between the finger 16 and thedisplay 12, such that the circle 18 may decrease in size as the finger16 approaches the display 12. In some implementations, an icon otherthan a circle is displayed at the position of the user's finger (e.g., asquare icon, an arrow icon, or a finger icon), and a size of the icon isa function of the distance between the user's finger 16 and the display12.

In some embodiments, the apparatus and methods use the optical sensor 10to allow a user to select or manipulate displayed items through contactwith the display 12. In such instances, the apparatus and methods maydetermine when the user's finger 16 or other pointer device is within athreshold touch distance of the display 12. In such an instance, theapparatus may consider the finger 16 to be in contact with the display,and identify a position on the surface of the display 12 correspondingto the contact. The apparatus and methods may then respond to the finger16 like any touch screen would respond. For example, if the usercontacts or taps a displayed button, the apparatus and methods mayexecute a task associated with the selection of the button. Likewise,when the user drags the finger 16 along the surface, within thethreshold touch distance, displayed images may be translated, rotated,expanded, or reduced in size, as desired. In one example, the user mayzoom in or out of an image (e.g., a 3D model) by contacting the displaywith two fingers and moving the fingers further apart or closertogether, respectively. Contact and movement with a single finger maycause the image to rotate. In general, the apparatus and methods arecapable of recognizing and responding to any contact or movement (e.g.,tapping, dragging, selecting, spinning, and/or any other gesture)provided by the user. The threshold touch distance may be, for example,less than about 1 inch, less than about 0.5 inches, or less than about0.25 inches.

In certain implementations, a single optical sensor may have difficultydetecting the position of a finger or other pointer device on or nearthe display. For example, if the user places one finger behind anotherfinger along a detection path of the sensor, a single optical sensor mayhave difficulty detecting the obscured finger. Likewise, with displaysthat are large (e.g., greater than 40 inches, measured diagonally), asingle optical sensor may have difficulty accurately detecting theposition of the finger over the entire surface of the display. In suchinstances, it may be desirable to provide more than one optical sensorfor the display. For example, referring to FIG. 2, two optical sensors20 may be positioned near adjacent corners of a display 22. In oneembodiment, the two optical sensors 20 are oriented in directions thatare substantially orthogonal to one another. With this two-sensorconfiguration, when one optical sensor is unable to detect an obscuredfinger, the other optical sensor is generally able to detect the finger.

Referring to FIG. 3, some embodiments of the invention may utilize fouroptical sensors 24, with each optical sensor 24 positioned near a cornerof a display 26. This four-sensor configuration may be appropriate forlarge displays (e.g., greater than 40 inches, greater than 60 inches, orgreater than 80 inches) and/or when more than one user is interactingwith the display. The use of multiple optical sensors allows one sensorto detect a finger that is obscured in the view of another sensor orthat is too far from another sensor to detect accurately. The systemsand methods described herein may utilize, for example, 1, 2, 4, 6, 8, ormore optical sensors per display.

Referring to FIG. 4, in certain examples, an optical sensor includes acamera that may be used to position the optical sensor with respect to adisplay surface 30. For example, the camera may be used to establish ordefine a detection region 32 that lies within a threshold detectiondistance 34 from the display surface 30. The user may also use thecamera to establish or define a threshold touch line 36 corresponding tothe threshold touch distance, within which a user's finger 38 is deemedto be touching or contacting surface 30 of the display. Givenmeasurement accuracies and/or geometric constraints, it may be difficultfor the optical sensor to determine when the finger 38 is in actualcontact with the surface of the display. Use of the threshold touchdistance allows a close proximity between the finger 38 and the surface30 of the display to count as actual contact.

Referring to FIGS. 5-7, in certain embodiments, a calibration procedureis utilized to ensure that an optical sensor is able to detect thelocation of a user's finger 40 in front of an entire surface of adisplay 42. The calibration procedure may include placing the finger 40in front of two or more corners of the display 42. When the finger 40 isin front of a corner 44 of the display, a calibration utility 46 (e.g.,a software program) may be used to identify the particular corner wherethe finger 40 is located. In one embodiment, the calibration utility 46provides one or more targets (e.g., a sequence of circles) on thedisplay that the user contacts with the finger 40.

In certain embodiments, the apparatus and methods utilize a skeletaltracking mode in which the user stands several feet (e.g., 3 to 15 feet)in front of a display and performs gestures to interact with thedisplay. With the skeletal tracking mode, one or more optical sensorsmay be facing away from the display, towards the user, such that theapparatus recognizes the user's skeletal features (e.g., hands, arms,and face) and responds according to movement of those features. Forexample, when the user raises a hand and faces a palm of the hand towardthe display, the apparatus may provide an indication of the user's handposition (e.g., a cursor) on the display. The user can then move thehand to a position corresponding to a desired location on the display.The user may interact further with the display by opening or closing oneor more fingers on the hand. For example, the user may select an item(e.g., a menu option or a button) on the display by quickly closing andopening the fingers (e.g., a quick pinch). The user may manipulate anitem (e.g., translate a 2D image or rotate a 3D virtual object) on thedisplay by forming a first with the hand (e.g., to grab the item) andmoving the first according to the desired manipulation of the item. Inone embodiment, the user may manipulate a 3D virtual object by formingtwo fists and moving the fists together, as a rigid body. To zoom in orout of the 3D virtual model, the user can move the two fists furtherapart or closer together, respectively.

The skeletal mode may also be used to facilitate the reception and/orinterpretation of voice commands from the user. For example, when theuser performs a designated hand gesture, such as moving the hand nearthe user's mouth, the apparatus may recognize that the user is about toissue a voice command. The display may provide an indication (e.g., amessage that says the apparatus is listening) that the apparatus isready to receive the voice command. Upon receipt of the voice command,the apparatus may take an action associated with the command.

The skeletal mode may also be used to provide messages to the userregarding the position of the user's body. For example, the apparatusmay provide a message to inform the user that the user's body and/orhead are not facing the display. Such messages may help the user obtainthe proper body position and/or let the user know when the user's bodyis not being detected.

In various embodiments, the optical sensors described herein are used todetermine (i) a distance between a user's finger (or other pointerdevice) and a display, and (ii) a position of the user's finger on or infront of the display. Referring to FIG. 8, an optical sensor 50 maymeasure a separation distance S, a radial distance R, and an angularposition θ of a user's finger 52 relative to a display 54. Theseparation distance S is a distance between a tip of the user's finger52 and a closest point P on the surface of the display 54 (e.g., in adirection normal to the display 54). The radial distance R is a distancebetween the optical sensor 50 and the closest point P. The radialdistance R may be measured, for example, using a depth sensorincorporated into the optical sensor 50. The angular position θ is anangle between (i) a centerline 56 or center axis of the optical sensor50 along the display 54, and (ii) a line between the optical sensor 50and closest point P along the display 54. The separation distance S andthe angular position θ may be measured, for example, using a cameraincorporated into the optical sensor 50. A scale factor may be used toadjust the measured separation distance S to account for the radialdistance R between the user's finger 52 and the optical sensor 50. Forexample, when the user's finger 52 is far from the optical sensor 50,the separation distance S may be larger than it appears from theperspective of a camera within the optical sensor 50. In such aninstance, the separation distance S may be increased according to theradial distance R, using the scale factor.

In some implementations, the radial distance R, the angular position θ,and the separation distance S are coordinates in a polar coordinatesystem, with the optical sensor 50 located at the origin of the polarcoordinate system. The polar coordinates may be transformed to Cartesiancoordinates (i.e., x, y, z coordinates) for the display 54, usingtransformation techniques known by those of ordinary skill in the art.

Referring to FIG. 9, in certain embodiments, a method 60 is provided forconverting a display into a touch display. At step 62, a signal isreceived from at least one optical sensor arranged proximate the displayand having at least one depth sensor. The signal is used to determine(step 64) a distance between a pointer device and a surface of thedisplay (e.g., the separation distance S). The signal is also used todetermine (step 66) a position of the pointer device on (or adjacent to)the surface of the display (e.g., the radial distance R and the angularposition θ). When the distance is less than a threshold detectiondistance and greater than a threshold touch distance, an indication maybe provided (step 68) on the display that the position of the pointerdevice has been determined. For example, a circle or other icon may bedisplayed at the pointer device position on the display. When thedistance is less than the threshold touch distance, an action is taken(step 70) that is indicative of contact between the pointer device andthe surface of the display. In some instances, step 68 is omitted fromthe method 60, such that no indication of the position may be providedon the display until the distance is less than the threshold touchdistance.

Embodiments of the apparatus and methods described herein may utilize acomputer system, which may include a general purpose computing device inthe form of a computer including a processor or processing unit, asystem memory, and a system bus that couples various system componentsincluding the system memory to the processing unit.

Computers typically include a variety of computer readable media thatcan form part of the system memory and be read by the processing unit.By way of example, and not limitation, computer readable media maycomprise computer storage media and communication media. The systemmemory may include computer storage media in the form of volatile and/ornonvolatile memory such as read only memory (ROM) and random accessmemory (RAM). A basic input/output system (BIOS), containing the basicroutines that help to transfer information between components, such asduring start-up, is typically stored in ROM. RAM typically contains dataand/or program modules that are immediately accessible to and/orpresently being operated on by processing unit. The data or programmodules may include an operating system, application programs, otherprogram modules, and program data. The operating system may be orinclude a variety of operating systems such as Microsoft Windows®operating system, the Unix operating system, the Linux operating system,the Mac OS operating system, Google Android operating system, Apple iOSoperating system, or another operating system or platform.

At a minimum, the memory includes at least one set of instructions thatis either permanently or temporarily stored. The processor executes theinstructions that are stored in order to process data. The set ofinstructions may include various instructions that perform a particulartask or tasks. Such a set of instructions for performing a particulartask may be characterized as a program, software program, software,engine, module, component, mechanism, or tool.

The system may include a plurality of software processing modules storedin a memory as described above and executed on a processor in the mannerdescribed herein. The program modules may be in the form of any suitableprogramming language, which is converted to machine language or objectcode to allow the processor or processors to read the instructions. Thatis, written lines of programming code or source code, in a particularprogramming language, may be converted to machine language using acompiler, assembler, or interpreter. The machine language may be binarycoded machine instructions specific to a particular computer.

Any suitable programming language may be used in accordance with thevarious embodiments of the invention. Illustratively, the programminglanguage used may include assembly language, Basic, C, C++, C#, CSS,HTML, Java, SQL, Perl, Python, Ruby and/or JavaScript, for example.Further, it is not necessary that a single type of instruction orprogramming language be utilized in conjunction with the operation ofthe system and method of the invention. Rather, any number of differentprogramming languages may be utilized as is necessary or desirable.

Also, the instructions and/or data used in the practice of the inventionmay utilize any compression or encryption technique or algorithm, as maybe desired. An encryption module might be used to encrypt data. Further,files or other data may be decrypted using a suitable decryption module.

The computing environment may also include otherremovable/non-removable, volatile/nonvolatile computer storage media.For example, a hard disk drive may read or write to non-removable,nonvolatile magnetic media. A magnetic disk drive may read from orwrites to a removable, nonvolatile magnetic disk, and an optical diskdrive may read from or write to a removable, nonvolatile optical disksuch as a CD-ROM or other optical media. Other removable/non-removable,volatile/nonvolatile computer storage media that can be used in theexemplary operating environment include, but are not limited to,magnetic tape cassettes, flash memory cards, digital versatile disks,digital video tape, solid state RAM, solid state ROM, Storage AreaNetworking devices, solid state drives, and the like. The storage mediaare typically connected to the system bus through a removable ornon-removable memory interface.

The processing unit that executes commands and instructions may be ageneral purpose computer, but may utilize any of a wide variety of othertechnologies including a special purpose computer, a microcomputer,mini-computer, mainframe computer, programmed micro-processor,micro-controller, peripheral integrated circuit element, a CSIC(Customer Specific Integrated Circuit), ASIC (Application SpecificIntegrated Circuit), a logic circuit, a digital signal processor, aprogrammable logic device such as an FPGA (Field Programmable GateArray), PLD (Programmable Logic Device), PLA (Programmable Logic Array),RFID integrated circuits, smart chip, or any other device or arrangementof devices that is capable of implementing the steps of the processes ofthe invention.

It should be appreciated that the processors and/or memories of thecomputer system need not be physically in the same location. Each of theprocessors and each of the memories used by the computer system may bein geographically distinct locations and be connected so as tocommunicate with each other in any suitable manner. Additionally, it isappreciated that each of the processor and/or memory may be composed ofdifferent physical pieces of equipment.

A user may enter commands and information into the systems that embodythe invention through a user interface that includes input devices suchas a keyboard and pointing device, commonly referred to as a mouse,trackball or touch pad. Other input devices may include a microphone,joystick, game pad, satellite dish, scanner, voice recognition device,keyboard, touch screen, toggle switch, pushbutton, or the like. Theseand other input devices are often connected to the processing unitthrough a user input interface that is coupled to the system bus, butmay be connected by other interface and bus structures, such as aparallel port, game port or a universal serial bus (USB).

The systems that embody the invention may communicate with the user vianotifications sent over any protocol that can be transmitted over apacket-switched network or telecommunications network. By way ofexample, and not limitation, these may include SMS messages, email(SMTP) messages, instant messages (GChat, AIM, Jabber, etc.), socialplatform messages (Facebook posts and messages, Twitter direct messages,tweets, retweets, etc.), and mobile push notifications (iOS, Android).

One or more monitors or display devices may also be connected to thesystem bus via an interface. In addition to display devices, computersmay also include other peripheral output devices, which may be connectedthrough an output peripheral interface. The computers implementing theinvention may operate in a networked environment using logicalconnections to one or more remote computers, the remote computerstypically including many or all of the elements described above.

Although internal components of the computer are not shown, those ofordinary skill in the art will appreciate that such components and theinterconnections are well known. Accordingly, additional detailsconcerning the internal construction of the computer need not bedisclosed in connection with the present invention.

It is understood that the methods and systems described above maycontain software and hardware connected to the Internet via a network.Computing devices are capable of communicating with each other via theInternet, and it should be appreciated that the various functionalitiesof the components may be implemented on any number of devices.

The invention may be practiced using any communications network capableof transmitting Internet protocols. A communications network generallyconnects a client with a server, and in the case of peer to peercommunications, connects two peers. The communication may take place viaany media such as standard telephone lines, LAN or WAN links (e.g., T1,T3, 56 kb, X.25), broadband connections (ISDN, Frame Relay, ATM),wireless links (802.11, Bluetooth, 3G, CDMA, etc.), and so on. Thecommunications network may take any form, including but not limited toLAN, WAN, wireless (WiFi, WiMAX), near-field (RFID, Bluetooth). Thecommunications network may use any underlying protocols that cantransmit Internet protocols, including but not limited to Ethernet, ATM,VPNs (PPPoE, L2TP, etc.), and encryption (SSL, IPSec, etc.)

The invention may be practiced with any computer system configuration,including hand-held wireless devices such as mobile phones or personaldigital assistants (PDAs), multiprocessor systems, microprocessor-basedor programmable consumer electronics, minicomputers, mainframecomputers, computers running under virtualization, etc.

The invention may also be practiced in distributed computingenvironments where tasks are performed by remote processing devices thatare linked through a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

The invention's data store may be embodied using any computer datastore, including but not limited to, relational databases,non-relational databases (NoSQL, etc.), flat files, in memory databases,and/or key value stores. Examples of such data stores include the MySQLDatabase Server or ORACLE Database Server offered by ORACLE Corp. ofRedwood Shores, Calif., the PostgreSQL Database Server by the PostgreSQLGlobal Development Group of Berkeley, Calif., the DB2 Database Serveroffered by IBM, Mongo DB, Cassandra, or Redis.

The terms and expressions employed herein are used as terms andexpressions of description and not of limitation, and there is nointention, in the use of such terms and expressions, of excluding anyequivalents of the features shown and described or portions thereof. Inaddition, having described certain embodiments of the invention, it willbe apparent to those of ordinary skill in the art that other embodimentsincorporating the concepts disclosed herein may be used withoutdeparting from the spirit and scope of the invention. The features andfunctions of the various embodiments may be arranged in variouscombinations and permutations, and all are considered to be within thescope of the disclosed invention. Accordingly, the described embodimentsare to be considered in all respects as only illustrative and notrestrictive. Furthermore, the configurations, materials, and dimensionsdescribed herein are intended as illustrative and in no way limiting.Similarly, although physical explanations have been provided forexplanatory purposes, there is no intent to be bound by any particulartheory or mechanism, or to limit the claims in accordance therewith.

What is claimed is:
 1. An apparatus for converting a display into a touch display, the apparatus comprising: at least one optical sensor arranged proximate the display and substantially within a plane defined by a surface of the display, the at least one optical sensor comprising at least one depth sensor and being oriented in a direction parallel to the surface of the display; and a processor configured to receive a signal from the at least one optical sensor, the processor configured to execute instructions for: (i) determining a first distance between at least one pointer device and a surface of the display, based on the signal; (ii) determining a position of the at least one pointer device on the surface of the display, based on the signal; and (iii) determining that the first distance is less than a threshold touch distance and, in response thereto, taking an action indicative of contact between the at least one pointer device and the surface of the display, wherein the at least one depth sensor comprises a laser projector configured to determine a second distance between the at least one optical sensor and the at least one pointer device, along the surface of the display.
 2. The apparatus of claim 1, wherein the at least one optical sensor further comprises at least one camera.
 3. The apparatus of claim 1, wherein the at least one optical sensor comprises a first optical sensor oriented in a first direction, and a second optical sensor oriented in a second direction, the first direction being substantially orthogonal to the second direction, the first and second directions being substantially parallel to the surface of the display.
 4. The apparatus of claim 3, wherein the at least one optical sensor further comprises a third optical sensor oriented in a third direction, and a fourth optical sensor oriented in a fourth direction, the third direction being substantially orthogonal to the fourth direction, the third and fourth directions being substantially parallel to the surface of the display.
 5. The apparatus of claim 1, wherein the at least one pointer device comprises two or more pointer devices, and the processor is configured to determine a position of each pointer device on the surface of the display.
 6. The apparatus of claim 1, wherein the at least one pointer device comprises at least one finger of a user.
 7. The apparatus of claim 1, wherein determining a position comprises determining a closest point to the at least one pointer device on the surface of the display.
 8. The apparatus of claim 1, wherein the threshold touch distance is less than about 1 inch.
 9. The apparatus of claim 1, wherein the processor is further configured to execute instructions for determining that the first distance is less than a threshold detection distance and greater than the threshold touch distance and, in response thereto, providing an indication on the display that the position of the at least one pointer device has been determined, the indication comprising adjusting an image on the display in a vicinity of the position of the at least one pointer device on the surface of the display.
 10. The apparatus of claim 9, wherein the threshold detection distance is less than about 6 inches.
 11. The apparatus of claim 1, wherein taking an action indicative of contact between the at least one pointer device and the surface of the display comprises an action selected from the group consisting of: selecting an item, translating an image, expanding an image, shrinking an image, and rotating a virtual object.
 12. The apparatus of claim 1, wherein determining the first distance between the at least one pointer device and the surface of the display comprises calculating a scale factor based on the second distance between the at least one optical sensor and the at least one pointer device.
 13. A method of converting a display into a touch display, the method comprising: receiving a signal from at least one optical sensor arranged proximate the display and substantially within a plane defined by a surface of the display, the at least one optical sensor comprising at least one depth sensor and being oriented in a direction parallel to the surface of the display; determining a first distance between at least one pointer device and a surface of the display, based on the signal; determining a position of the at least one pointer device on the surface of the display, based on the signal; and determining that the first distance is less than a threshold touch distance and, in response thereto, taking an action indicative of contact between the at least one pointer device and the surface of the display, wherein the at least one depth sensor comprises a laser projector configured to determine a second distance between the at least one optical sensor and the at least one pointer device, along the surface of the display.
 14. The method of claim 13, wherein the at least one optical sensor further comprises at least one camera.
 15. The method of claim 13, wherein the at least one optical sensor comprises a first optical sensor oriented in a first direction, and a second optical sensor oriented in a second direction, the first direction being substantially orthogonal to the second direction, the first and second directions being substantially parallel to the surface of the display.
 16. The method of claim 15, wherein the at least one optical sensor further comprises a third optical sensor oriented in a third direction, and a fourth optical sensor oriented in a fourth direction, the third direction being substantially orthogonal to the fourth direction, the third and fourth directions being substantially parallel to the surface of the display.
 17. The method of claim 13, wherein the at least one pointer device comprises two or more pointer devices, and determining a position of the least one pointer device comprises determining a position of each pointer device on the surface of the display.
 18. The method of claim 13, wherein the at least one pointer device comprises at least one finger of a user.
 19. The method of claim 13, wherein determining a position comprises determining a closest point to the at least one pointer device on the surface of the display.
 20. The method of claim 13, wherein the threshold touch distance is less than about 1 inch.
 21. The method of claim 13, further comprising determining that the first distance is less than a threshold detection distance and greater than the threshold touch distance and, in response thereto, providing an indication on the display that the position of the at least one pointer device has been determined, wherein providing the indication on the display comprises adjusting an image on the display in a vicinity of the position of the at least one pointer device on the surface of the display.
 22. The method of claim 21, wherein the threshold detection distance is less than about 6 inches.
 23. The method of claim 13, wherein taking an action indicative of contact between the at least one pointer device and the surface of the display comprises an action selected from the group consisting of: selecting an item, translating an image, expanding an image, shrinking an image, and rotating a virtual object.
 24. The method of claim 13, wherein determining the first distance between the at least one pointer device and the surface of the display comprises calculating a scale factor based on the second distance between the at least one optical sensor and the at least one pointer device. 